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CLAIMS 

1. An apparatus for generating stereo or motion image information, 
5 comprising: 

at least one imaging sensor for producing intensity images 
represented by an array N x IVI of numbers, wliere each number corresponds 
to the intensity of light falling on a particular array position; 

a rectification module for mapping an original intensity image to a 
10 rectified image having substantially horizontal epipolar lines; 
a feature extraction module; 

a correlation module for comparing feature values over a window of 

size XCORR x YCORR in a first feature image to a similar window in a 

second feature image, as displaced by a disparity; 
15 a summing module for determining a confidence value by summing 

an interest operator over a correlation window; and 

means for calculating a disparity result image by performing an 

extrema extraction to find a minimum summed correlation value 

corresponding to the disparity of a best match; 
20 wherein an image of disparity values is produced having 

approximately the size of said original images, and wherein each pixel in 

said disparity image is the disparity of a corresponding pixel in said first 

intensity image. 
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2. The apparatus of Claim 1 , further comprising: 

means for eliminating disparity results having a low-confidence on the 
basis of thresholded confidence values produced by said summing module; 

a window summation buffer; and 
5 means for performing a left-right consistency check on said window 

summation buffer. 

3. The apparatus of Claim 1 , wherein two images to be correlated may 
come either from two different cameras separated spatially that capture 

10 images at the same time to produce a stereo image, or from the same 
camera capturing two successive images at different times to produce a 
motion disparity image. 

4. The apparatus of Claim 1 , wherein said rectification and feature 
15 extraction module separately processes each of two or more intensity 

images. 

5. The apparatus of Claim 4, wherein features are computed on rectified 
images. 

20 

6. The apparatus of Claim 1, wherein features are computed using a 
Laplacian of Gaussian ("LOG") operator. 

7. The apparatus of Claim 1, wherein said rectification module and said 
25 feature extraction module are combined, such that rectification and feature 

extraction are performed in a single operation. 
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8. The apparatus of Claim 1 , wherein said correlation module operates 
on successive lines of said feature images, such that it is necessary to buffer 
only YCORR+1 lines of said feature image, where YCORR is the height of a 

5 correlation window. 

9. The apparatus of Claim 1 , wherein said correlation module operates 
on each feature image line after said line is computed by said rectification 
module and said feature extraction module, such that rectification and 

10 feature extraction proceed in parallel with correlation. 

10. The apparatus of Claim 2, wherein said correlation module operates 
on successive lines of said feature images, updating said window 
summation buffer. 

15 

11. The apparatus of Claim 2, wherein said window summation buffer has 
size N X (D+1), where D is the number of different disparities that are 
checked for each pixel in said feature images, where for each disparity 0 <= 
d < D there is a line of size N in said window summation buffer, where each 

20 value in said line is the correlation of said window centered on a 
corresponding pixel in said first feature image to said window centered on a 
corresponding pixel offset by the disparity d in said second feature image. 

12. The apparatus of Claim 1 1 , wherein the disparity offset in said second 
25 feature image is along a same horizontal line as for said first feature image 

for stereo. 
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13. The apparatus of Claim 11, wherein the disparity offset in said first 
feature image is in a local horizontial and vertical neighborhood around a 
corresponding pixel in said second feature image for motion. 

5 

14. The apparatus of Claim 2, further comprising: 

means for determining a confidence value by summing an interest 
operator over the same correlation window at the same time as said 
correlation step is operating. 

10 

15. The apparatus of Claim 14, wherein results of an interest operator for 
each new line are stored in one line of said window summation buffer. 

16- The apparatus of Claim 2, wherein said means for eliminating 
1 5 disparity produces an interpolated sub-pixel disparity. 

17, The apparatus of Claim 7, wherein said rectification and feature 
extraction module applies an LOG operator to the neighborhood of (ri , rj ) 
and stores a result at (i,j). 

20 

18. The apparatus of Claim 17, wherein said rectification and feature 
extraction module uses sub-pixel mapping between a rectified image and a 
corresponding original image, where the original image coordinates (ri , rj ) 
are real numbers. 

25 
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19. The apparatus of Claim 17. wherein said rectification and feature 
extraction module calculates coefficients for fractionally-shifted operators F, 
where L(x,y) is a LOG function giving a coefficient at (x,y), and where the 
operators are given by the functions L(x-a, y-b), where a < F and b < F. 

20. The apparatus of Claim 2, wherein YCORR lines are computed in 
each feature image, where for each disparity d < D there is a line of length N- 
XCORR + 1 in said window summation buffer, where each entry in said line 
holds the sum of correlations over corresponding windows in said feature 
images. 

21. The apparatus of Claim 20, wherein said window summation buffer 
computes a sum by first correlating corresponding pixels in each of said 
feature image windows and then summing the results. 

22. The apparatus of Claim 21, wherein correlation between said pixels is 
the square of the difference of their values or the absolute value of the 
difference of their values. 

23. The apparatus of Claim 21, wherein the sum in said window 
summation buffer is used to produce one line of a disparity image, where a 
new line is computed for each of said feature images, said window 
summation buffer is updated to reflect said new values, and another disparity 
image line is produced. 
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24. A method for generating stereo or motion Image Information, 
comprising the steps of: 

. rectifying an Intensity Image; 

computing a feature Image from said intensity Image. 

comparing said feature image values over a window of size XCORR x 
YCORR for a first feature image to a similar window for a second feature 
Image, displaced by a disparity; and 

calculating a disparity result Image. 

25. A method for generating stereo or motion Image Information, 
comprising the steps of: 

producing Intensity Images with at least one Imaging sensor, said 
intensity images represented by an array N x M of numbers, where each 
number corresponds to the Intensity of light falling on a particular array 
position; 

mapping an original intensity Image to a rectified image having 
substantially horizontal epipolar lines; 

extracting features from said rectified Image; 

comparing feature values over a window of size XCORR x YCORR in 
a first feature Image to a similar window in a second feature Image, as 
displaced by a disparity; 

determining a confidence value by summing an interest operator over 
a correlation window; and 

calculating a disparity result Image by performing an extrema 
extraction to find a minimum summed correlation value corresponding to the 
disparity of a best match; 
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wherein an image of disparity values is produced having 
approximately the size of said original images, and wherein each pixel in 
said disparity image is the disparity of a corresponding pixel in said first 
intensity image. 

5 

26. The method of Claim 25, further comprising the steps of: 
eliminating disparity results having a low-confidence on the basis of 

thresholded confidence values produced by said determining step; and 

performing a left-right consistency check on said window summation 

10 buffer. 

27. The method of Claim 25, wherein two images to be correlated may 
come either from two different cameras separated spatially that capture 
images at the same time to produce a stereo imagef, or from the same 

1 5 camera capturing two successive images at different times to produce a 
motion disparity image. 

28. The method of Claim 25, wherein said rectification and feature 
extraction steps separately processes each of two or more intensity images, 

20 

29. The method of Claim 28, wherein features are computed on rectified 
images. 

30. The method of Claim 25, wherein features are computed using a 
25 Laplacian of Gaussian ("LOG") operator 
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31. The method of Claim 25, wherein said rectification step and said 
feature extraction step are combined, such that rectification and feature 
extraction are performed in a single operation. 

5 32. The method of Claim 25, wherein said con-elation step operates on 
successive lines of said feature images, such that it is necessary to buffer 
only YCORR+1 lines of said feature image, where YCORR is the height of a 
correlation window, 

1 0 33. The method of Claim 25, wherein said correlation step operates on 
each feature image line after said line is computed by said rectification step 
and said feature extraction step, such that rectification and feature extraction 
proceed in parallel with correlation. 

1 5 34. The method of Claim 26, wherein said con-elation step operates on 
successive lines of said feature images, updating said window summation 
buffer. 

35. The method of Claim 26, wherein said window summation buffer has 
20 size N x (D+1), where D is the number of different disparities that are 
checked for each pixel in said feature images, where for each disparity 0 <= 
d < D there is a line of size N in said window summation buffer, where each 
value in said line is the correlation of said window centered on a 
corresponding pixel in said first feature image to said window centered on a 
25 corresponding pixel offset by the disparity d in said second feature image. 
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36. The method of Claim 35, wherein the disparity offset in said second 
feature image is along a same horizontal line as for said first feature image 
for stereo. 

5 37. The method of Claim 35, wherein the disparity offset in said first 
feature image is in a local horizontal and vertical neighborhood around a 
corresponding pixel in said second feature image for motion. 

38. The method of Claim 26, further comprising the step of: 

1 0 determining a confidence value by summing an interest operator over 

the same correlation window at the same time as said correlation step is 
operating. 

39. The method of Claim 38, wherein results of an interest operator for 
1 5 each new line are stored in one line of said window summation buffer. 

40. The method of Claim 26, wherein said eliminating disparity step 
produces an Interpolated sub-pixel disparity. 

20 41. The method of Claim 31, wherein said rectification and feature 
extraction step applies an LOG operator to the neighborhood of (ri , rj ) and 
stores a result at (i,j). 

42. The method of Claim 41, wherein said rectification and feature 
25 extraction step uses sub-pixel mapping between a rectified image and a 
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corresponding original image, where the original image coordinates (ri , rj ) 
are real numbers. 

43. The method of Claim 41, wherein said rectification and feature 
5 extraction step calculates coefficients for fractionally-shifted operators F, 

where L(x,y) is a LOG function giving a coefficient at (x,y), and where the 
operators are given by the functions L{x-a, y-b), where a < F and b < F. 

44, The method of Claim 26, wherein YCORR lines are computed in each 
10 feature image, where for each disparity d < D there is a line of length N- 

XCORR + 1 in said window summation buffer, where each entry in said line 
holds the sum of correlations over corresponding windows in said feature 
images. 

15 45. The method of Claim 44, wherein said window summation buffer 
computes a sum by first correlating corresponding pixels in each of said 
feature image windows and then summing the results. 

46. The method of Claim 45, wherein correlation between said pixels is 
20 the square of the difference of their values. 

47. The method of Claim 45, wherein the sum in said window summation 
buffer is used to produce one line of a disparity image, where a new line is 
computed for each of said feature images, said window summation buffer is 

25 updated to reflect said new values, and another disparity image line is 
produced. 
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48. An apparatus for generating stereo or motion image information, 
comprising: 

one or more imagers; and 

a digital signal processor, said digital signal processor comprising: 
a module for rectifying an intensity image; 
a module for computing a feature image from said intensity 

Image. 

a module for comparing said feature image values over a 
window of size XCORR x YCORR for a first feature image to a similar window 
for a second feature image, displaced by a disparity; and 

a module for calculating a disparity result image. 
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